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ABSTRACT 

A  knowledge  acquisition  program  was  written  to  aid 
in  obtaining  knowledge  from  the  experts  concerning  en- 
demic populations  of  mountain  pine  beetle  in  lodgepole 
pine  forest.  An  application  expert  system  is  then  automa- 
tically generated  by  the  knowledge  acquisition  program 
that  contains  the  codified  base  of  expert  knowledge.  Data 
can  then  be  entered  into  the  expert  system  to  generate 
predictions. 


KEYWORDS:  Pinus  contorta,  Dendroctonus  ponderosae, 
knowledge  acquisition 

Expert  systems  (ES)  are  computer  programs  that  proc- 
ess information  in  ways  that  mimic  the  deductive  or  in- 
ductive reasoning  processes  of  a  human  expert  (Negotia 
1985).  Expert  systems  are  receiving  greater  attention  in 
the  area  of  natural  resources  as  tools  to  organize  existing 
knowledge  for  use  by  the  land  manager  or  research  scien- 
tist. This  paper  details  the  work  at  a  first-pass  attempt 
in  developing  an  ES  for  endemic  populations  of  mountain 
pine  beetle  (MPB)  (Dendroctonus  ponderosae  Hopkins 
[Coleoptera:  Scolytidae]),  in  lodgepole  pine  (Pinus  contorta 
Dougl.  var.  latifolia  Engelm.)  forests.  This  project  was 
undertaken,  in  part,  to  add  to  our  knowledge  about  how 
to  develop,  implement,  evaluate,  and  modify  expert  sys- 
tems in  our  area  of  expertise  (Stock  1987). 

Early  examples  of  expert  systems  have  been  in  such 
fields  as  mineral  exploration  where  human  experts  are 
scarce  and  the  cost  of  equipment  lying  idle  is  so  high  that 
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the  price  of  an  ES  can  be  recaptured  rapidly;  in  medicine 
where  extensive  written  records  on  the  best  known  ways 
to  diagnose  diseases  exist;  and  in  finance  and  accounting 
because  expert  knowledge  is  relatively  concrete  and  can 
be  easily  incorporated  into  a  knowledge  base  (Negotia 
1985). 

"Fuzziness"  due  to  the  ill-defined  nature  of  decision 
variables  and  processes  is  a  part  of  problem  solving 
in  many  fields,  including  natural  resources.  Where  de- 
cisions are  at  best,  semistructured  human  judgment  is 
required,  although  computer  processing  and  other  ana- 
lytical aids  are  valuable  in  discovering  patterns  as  well 
as  in  spotting  irregularities  and  other  inconsistencies 
that  are  important  to  understanding  processes  at  work. 
Expert  systems  enable  us  to  work  systematically  with 
concrete  knowledge  as  well  as  material  containing  a  high 
degree  of  expert  judgment.  At  any  stage  in  the  evolution 
of  an  ES,  it  might  best  be  regarded  as  an  adviser  or  source 
of  "second  opinion"  that  asks  necessary  questions  and 
gives  advice  within  a  context  of  current  but  changing 
state-of-the-art  knowledge  and  experience. 

Difficulties  encountered  in  the  development  and 
application  of  ES  technology  are  varied  and  include: 
(1)  selecting  topics  suitable  for  ES  development;  (2)  main- 
taining an  objective  perspective  necessary  for  recognizing 
ill-defined  characteristics  of  subject  matter  and  the 
patience  to  reformulate  them  during  the  knowledge 
acquisition/organization  process;  (3)  capturing  a  base 
of  knowledge  adequate  for  generating  valid  predictions; 
and  (4)  helping  end-users  adapt  selected  aspects  of  their 
work  to  maximize  the  benefits  of  the  system.  Moreover, 
the  system  designer  (knowledge  engineer)  must  assure 
experts  and  end-users  that  the  ES  is  not  final  although 
it  may  reach  a  stage  suitable  for  use  as  a  working  model 
in  the  field — one  that  can  be  further  modified  as  knowl- 
edge accumulates  or  as  inconsistencies  between  model 
predictions  and  actual  events  are  discovered. 
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PROJECT  OVERVIEW 


OBJECTIVES 


The  goal  of  this  work  is  to  devise  an  information/ 
decision  support  system  that  provides  expert  knowledge 
to  researchers  and  managers  in  a  user  friendly,  easily 
revised  format.  Our  ES  will  deal  with  endemic  (initial 
stages  prior  to  outbreak)  populations  of  MPB  in  lodgepole 
pine  forests  in  the  Intermountain  West.  This  work  will 
complement  the  current  research  being  conducted  by 
Forest  Service  scientists  with  the  mountain  pine  beetle 
project  in  Ogden,  UT.  We  anticipate  that  the  ES  gen- 
erated will  approach  that  of  a  "research  prototype" 
(Waterman  1986). 

During  the  past  20  years  the  MPB  project  in  Ogden 
has  developed  an  extensive  knowledge  base  about  the 
epidemic  phase  of  MPB  infestation.  These  contributions 
have  provided  a  good  understanding  of  the  dynamics  of 
epidemic  level  populations  in  lodgepole  pine  (Amman  and 
Cole  1983;  Cole  and  Amman  1980;  Cole  and  others  1985). 

A  hazard  rating  system  to  determine  stand  susceptibil- 
ity to  MPB  in  lodgepole  pine  forests  has  been  developed. 
Also,  researchers  realized  that  direct  control  of  outbreak 
level  populations  was  a  holding  action  at  best  and  di- 
rected attention  toward  silvicultural  means  of  suppress- 
ing MPB  under  outbreak  conditions.  These  conclusions 
emphasized  the  need  for  a  better  understanding  of  the 
dynamics  of  low  or  endemic  level  MPB  populations  that 
should  ultimately  lead  to  development  of  preventive 
strategies. 

Years  of  working  with  epidemic  MPB  populations  have 
given  rise  to  better  understanding  of  endemic  situations. 
Research  dealing  with  low  level  population  densities  has 
shown  the  beetle  is  sometimes  associated  with  diseased 
lodgepole  pine  trees,  particularly  those  having  armillaria 
root  disease  (Tkacz  and  Schmitz  1986)  and  commandra 
rust  (Rasmussen  1987).  The  role  of  these  associations 
and  other  leads  developed  from  initial  investigations  of 
low  level  MPB  populations  will  be  tested  in  a  study  of  the 
dispersion  of  low  level  MPB  populations  on  the  Medicine 
Bow  National  Forest  in  southeastern  Wyoming.  This  field 
effort  will  provide  a  framework  for  development  of  the  ES. 

This  paper  details  the  steps  followed  in  the  develop- 
ment of  several  expert  systems  for  field  testing: 

1.  Elicit  from  experts  (scientists  and  managers)  a  spe- 
cific objective  for  the  ES  model  (example:  to  predict  new 
tree  kills  for  individual  lodgepole  pine  stands.) 

2.  Elicit  knowledge  and  professional  judgment  neces- 
sary for  formulating  one  or  more  predictive  models. 

3.  Elicit  predictions  for  each  combination  of  conditions 
represented  in  each  model. 

4.  Elicit  an  estimate  of  certainty  associated  with  each 
prediction. 

5.  Convert  the  conceptual  ES  model(s)  into  an  opera- 
tional computerized  ES  program. 

6.  Generate  field  data  sheets  for  use  in  collecting  ap- 
propriate data  for  ES. 

7.  Verify  ES — see  how  well  the  predicted  mimics  real 
life. 


This  work  addresses  the  following  objectives: 

1.  To  develop  a  model  or  models  that,  in  the  opinion  of 
one  or  more  experts,  reflect  conditions  contributing  to  the 
initiation  of  outbreaks  from  within  stands  with  endemic 
populations  of  MPB. 

2.  To  assess  the  role  of  the  expert  system  in  facilitating 
the  generation,  clarification,  refinement,  and  verification 
of  scientific  knowledge. 

KNOWLEDGE  ACQUISITION 

We  acquire  specific  programming  objectives,  facts,  hy- 
pothesized relationships  among  facts,  predictions,  and 
estimates  of  certainty  of  prediction  by  extracting  infor- 
mation from  published  literature  and  from  experts  in  the 
field.  Because  much  of  the  knowledge  used  for  successful 
decision  making  in  natural  resources  has  evolved  through 
professional  experience,  and  because  ES  programming 
allows  us  to  incorporate  this  kind  of  qualitative,  heuristic 
reasoning,  we  view  skilled  practitioners  and  researchers 
as  major  sources  of  meaningful  information  for  model 
building.  We  capture  their  thought  processes  and  their 
knowledge  of  facts  and  how  they  interrelate  those  facts. 
This  enables  us  to  explain  within  the  program  how  pre- 
dictions or  conclusions  were  generated.  Capturing  knowl- 
edge from  experts  is  difficult  and  time  consuming  and 
something  commonly  referred  to  as  the  "knowledge  ac- 
quisition bottleneck."  The  main  problem  arises  from 
the  inability  of  experts  to  describe  their  own  reasoning 
processes — they  are  not  "expert"  at  articulating  what 
they  know  (Shapiro  1987). 

STRUCTURING  KNOWLEDGE 

While  knowledge  acquisition  has  received  little  system- 
atic research  (Hoffman  1987),  our  approach  to  cope  with 
this  problem  has  included:  (1)  interviewing  experts  about 
conditions  under  which  we  might  expect  MPB  buildup  to 
occur  as  well  as  when  we  would  expect  buildup  not  to 
occur;  (2)  watching  them  go  about  their  work  in  the  field; 
and  (3)  feeding  information  back  to  them  in  different 
formats  such  as  flowcharts,  maps,  three-dimensional 
graphics,  and  summary  printouts  of  their  responses. 

Two  kinds  of  knowledge  processing  programs  are  being 
used.  The  first,  a  knowledge  acquisition  program  written 
for  this  project,  enables  us  to  prototype  and  revise  infor- 
mation rapidly  as  experts  enter,  reorganize,  and  refine 
their  thoughts  in  response  to  feedback.  (In  future  studies 
we  hope  to  analyze  various  knowledge  acquisition  tech- 
niques and  compare  them  with  procedures  being  used  in 
this  investigation.)  This  approach  allows  us  the  means 
to  gather  knowledge  from  the  experts)  and  to  develop 
a  sound  knowledge  base  to  be  used  in  the  expert  system. 
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MODEL:  EXPERT 
A 


MODEL:  EXPERT 
B 


MODEL:  EXPERT 
C 


MODEL:  EXPERT 
D 


ONE  OR  MORE 
INTEGRATED 
MODELS  FOR 

FIELD  TESTING 


Figure  1 — How  models  are  integrated  from  different 
experts. 


built  by  working  directly  with  three  to  four  Forest 
Service  researchers  and  university  scientists  in  Utah 
and  Colorado  and  two  to  three  pest  management  special- 
ists in  Colorado,  Montana,  and  Utah.  Information  was 
collected  independently  from  each  individual.  The  infor- 
mation from  one  was  not  shared  with  the  others  until  all 
models  had  been  captured,  integrated  to  the  extent  pos- 
sible, and  sent  as  a  complete  set  of  protocols  to  the  origi- 
nal participants  and  other  specialists  for  review.  This 
process  was  designed  to  elicit  new  perspectives  as  well 
as  to  provide  a  basis  for  "collective  work"  (Winograd 
and  Flores  1987,  p.  158)  in  support  of  focused  scientific 
dialogue. 

The  interview-knowledge  acquisition  process  used  with 
the  initial  participants  is  currently  being  done  as  follows. 


The  second  program,  an  applications  expert  system, 
is  generated  automatically  by  the  first  and  contains  the 
codified  base  of  expert  knowledge.  Case  data  can  be  en- 
tered into  the  applications  ES  to  generate  predictions. 

Because  our  efforts  focus  specifically  on  techniques  for 
capturing  scientific  expertise  in  a  form  suitable  for  fur- 
ther scientific  interchange,  refinement,  and  field  verifica- 
tion, our  concern  is  with  linking  what  is  known  with  what 
is  less  certain  as  well  as  what  is  currently  incapable  of 
satisfactory  explanation. 

Moreover,  we  intend  to  use  this  process  to  capture  the 
perspectives  of  several  investigators  and  compare  one 
model  against  another  for  similarities  and  differences 
in  objectives,  factors,  factor  categories,  interrelationships, 
nature  of  predictions,  and  certainty  levels.  From  this  in- 
formation one  or  more  integrated  models  can  be  con- 
structed easily  in  a  form  suitable  for  rigorous  scientific 
verification  (fig.  1). 

SYSTEM  DEVELOPMENT 

The  dynamics  of  MPB  at  endemic  levels  has  received 
little  attention,  so  our  work  is  directed  at  capturing 
the  experiences  and  judgments  of  scientists  and  pest 
management  specialists  who  have  significant  experience 
in  following  the  progress  of  MPB  under  epidemic  con- 
ditions. Several  applications  expert  systems  have  been 


Interviewing,  Observing,  and 
Conceptual  Diagramming 

Each  scientist  and  pest  management  specialist  is 
briefed  in  general  on  what  the  model  is  being  designed 
to  explain  or  predict.  The  respondents  are  then  asked 
to  formulate  a  specific  objective  and  list  the  kinds  of  infor- 
mation (factors)  they  would  request  before  making  a  pre- 
diction about  whether  the  MPB  population  (future  tree 
kills  or  MPB  population  change)  would  increase,  remain 
relatively  static,  or  decline.  Also  elicited  were  appropriate 
response  categories  such  as  high,  moderate,  low,  or  iso- 
lated tree  kills  increasing  to  clumps  of  trees  killed,  clumps 
declining  to  scattered  individuals,  or  attacking/emerging 
adult  ratio  trend. 

Where  factors  themselves  must  be  predicted,  other  vari- 
ables are  listed.  For  example,  to  estimate  attack  success 
last  season,  it  may  be  necessary  to  make  judgments  about 
weather  variables  at  flight  time.  Figure  2A  represents 
the  process  of  working  backward  from  objective  to  factors 
and  relationships  during  modeling  sessions  with  the 
experts).  Figure  2B  illustrates  a  generalized  flowchart 
view  of  the  information  captured.  It  is  reviewed  and  re- 
fined repeatedly  until  it  reflects  the  expert's  views  of  proc- 
esses at  work.  Other  kinds  of  graphics  (fig.  3)  are  used  to 
trigger  the  expert's  thought  processes. 
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Continue 
capturing  factor 
sets  until  model 
reflects  experts's 
view  of  factors 
and  relationships 
influencing  the 
status  of  X,  A1,  etc. 


B 


PHASE  3:  Elicit 
B-level  factors  and 

subcategories 
required  by  expert 

to  predict  or 
explain  status  of 
factor  A1. 


PHASE  2:  Elicit 
A-level  factors  and 

subcategories 
required  by  expert 
to  predict  or 
explain  the  status  of 
phenomenon  X. 


FACTOR  B1 

Subcategory  1 
Subcategory  2 


FACTOR  B2 

Subcategory  1 
Subcategory  2 
Subcategory  3 


FACTOR  A1 

Subcategory  1 
Subcategory  2 
Subcategory  3 


FACTOR  A2 

Subcategory  1 
Subcategory  2 


FACTOR  A3 

Subcategory  1 
Subcategory  2 
Subcategory  3 
Subcategory  4 
Subcategory  5 


PHASE  1 :  Elicit 
from  expert  the 
objective  or  nature  of 
explanation  or 
prediction  to  be 
made  including 
expected  outcome/ 
categories. 


POSSIBLE  STATES  OF 
PHENOMENON  X 

— Outcome  A  or 
— Outcome  B  or 
— Outcome  C  or 
— Outcome  D  etc. 


Figure  2 — (A)  Knowledge  acquisition  phases  including  factors  and  relationships.  (B) 
Feedback  to  experts  including  the  structure  of  the  acquired  knowledge  base — factors 
and  relationships. 
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Figure  3 — Graphics  used  to  trigger  expert's  thought  processes. 
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Generating  Predictions  for  Factor 
Combinations 

Once  the  diagram  is  finished,  factors  and  subcategories 
are  entered  into  the  knowledge  acquisition/processing 
program  and  a  printout  is  generated  listing  all  combina- 
tions of  factors/subcategories  given  by  the  respondent 
(fig.  4). 

The  respondent  then  enters  three  types  of  information 
on  the  printout  for  each  factor  combination: 

1.  Predicted  population  change. 

2.  A  numerical  value  that  represents  the  degree  of  cer- 
tainty the  respondent  has  in  the  prediction. 

3.  One  or  more  comments  that  qualify  or  otherwise 
explain  the  rationale  for  the  prediction.  Published  refer- 
ence citations  that  support  or  refute  the  prediction  may 


be  entered  as  well.  Qualifying  comments  can  be  edited 
and  added  to  the  final  applications  program  for  recall  at 
run  time  by  users  who  want  more  information  about  the 
basis  for  questions  asked  the  program. 

Experience  suggests  it  is  easier  for  the  respondent 
to  enter  responses  directly  on  the  printout  rather  than 
enter  into  the  computer.  Predictions,  certainty  factors, 
and  comments  can  be  quickly  added  to  the  program 
knowledge  base  later  by  the  system  designer. 

Generating  a  Data  Collection  Form 

A  field  data  entry  form  is  generated  automatically  for 
record  entry.  It  mirrors  the  factors  and  factor  categories 
reflected  in  the  mental  model  as  it  has  been  designed  by 
the  respondent.  As  a  result,  provision  is  made  to  ensure 
that  data  required  to  verify  the  model(s)  are  collected. 


OBJECTIVE:  To  predict/explain  post-flight  tree  kill  pattern. 

TOTAL  RULES  ENTERED:  4    RULE  NUMBER:  1 
PREDICTION:  ->  Q  p. 
Estimated  certainty  value  [0-1 0  with  0  =  none]:  >£/ 


Case  conditions 


tree_kill_pattem  _past_season 

over_wintering_brood_level 

attack_success_at_flight_time 


single  trees  widely  scattered 

moderate 

moderate 


Predictions, 
certainty  values, 
and  comments 
entered  here. 


COMMENT  1: 


COMMENT  2: 


OBJECTIVE:  To  predict/explain  post-flight  tree  kill  pattern. 

TOTAL  RULES  ENTERED:  4    RULE  NUMBER:  2 
PREDICTION:  ->  © 

Estimated  certainty  value  [0-10  with  0  =  nonel:  w 


Case  conditions 


tree_kill_pattern_past_season 

over_wintering_brood_level 

attack_success_at_flightjime 


single  trees  widely  scattered 
low 

moderate 


COMMENT  1:  (3j 


COMMENT  2: 


Figure  4 — Sample  printout  of  knowledge  acquisition/processing  program  that 
includes  all  combinations  of  factors/subcategories. 
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Generating  Predictions  From  Field 
Data 

Field  data  are  processed  through  the  applications 
ES,  and  expected  outcomes  or  predictions  are  displayed. 
For  research,  the  results  can  be  matched  against  actual 
events — a  key  step  in  verification.  For  management,  the 
predictions  can  be  used  to  plan  management  actions. 

RESULTS  AND  DISCUSSION 

At  this  stage  of  the  investigation  we  limit  our  descrip- 
tion of  results  to  two  sets  of  general  statements  or  propo- 
sitions that  seem  to  characterize  the  substantive  and 
methodological  findings.  These  propositions  will  be  the 
focus  of  further  investigation  in  both  the  substantive  area 
of  investigation  (MPB  dynamics)  and  knowledge  acquisi- 
tion methodology. 

I.  Behavior  of  Endemic  MPB 
Populations  in  Lodgepole  Pine 

1.  There  are  areas  of  agreement  as  well  as  disagree- 
ment among  the  researchers  and  specialists  we  have 
interviewed  regarding  priorities  for  investigation  (re- 
search objectives),  factors,  interrelationships,  predicted 
outcomes,  and  levels  of  certainty.  Models  being  charted 
and  programmed  will  provide  a  framework  for  focused 
discussion  on  priorities  for  further  investigation  as  well 
as  strategies  for  testing  and  validation. 

2.  A  scientific  model  of  population  dynamics  will  differ 
from  a  management-oriented  model.  The  scientific  model 
that  emerges  will  be  a  more  complete  description  of  how 
endemic  populations  change  over  time  in  response  to 
short-term  as  well  as  long-term  population,  stand,  and 
weather  factors.  The  management  model  will  emphasize 
long-term  factors,  particularly  projected  stand  conditions, 
because  of  the  need  to  plan  remedial  actions  3  to  6  years 
ahead  of  actual  events.  Short-term  variation  in  weather, 
tree,  or  stand  factors  is  likely  to  be  less  relevant  in  a  man- 
agement context. 

II.  Perceived  Contribution  of  Expert 
Systems  Development  in  the  Scientific 
Process 

1.  There  are  competing  models  for  endemic  population 
dynamics,  and  this  variation  in  perspective  serves  as  a 
barrier  to  the  interchange  of  ideas.  Emphasizing  one 
model  and  discrediting  the  possibilities  offered  by  others 
complicates  the  research  process.  However,  the  process 
being  used  in  this  investigation  promotes  communication 
among  researchers  and  yields  a  final  model  that  is  better 
because  of  the  different  perspectives  contributed  by  each 
participant. 

2.  The  possibility  of  rapid  update,  refinement,  expres- 
sion of  certainty  level,  and  guidance  for  future  verification 


that  is  designed  into  the  knowledge  processing  system 
makes  professional  speculation  within  a  scientific  context 
a  tolerable  activity.  Scientists  are  willing  to  describe 
their  views  and  to  offer  predictions  even  though  they 
must  extrapolate  from  related  but  not  identical  experi- 
ences or  systematic  investigations. 

3.  The  potential  for  simultaneous  testing  of  competing 
models  is  a  positive  prospect.  Scientists  are  showing 
interest  in  how  the  models  of  their  colleagues  will  com- 
pare. Where  differences  exist,  alternative  ES  models  can 
be  constructed  and  tested  simultaneously.  It  is  not  neces- 
sary to  achieve  consensus  on  one  model  initially.  Rather, 
the  objective  is  to  formulate  competing  models  so  they  can 
be  subjected  to  valid  testing. 

4.  Responses  from  experts  to  feedback  during  system 
development  fall  into  several  categories:  (a)  substance 
of  the  feedback  is  incorrect — the  system  developer  misin- 
terpreted the  information;  (b)  feedback  causes  the  respon- 
dent to  think  about  additional  factor  combinations  and 
suggests  other  avenues  for  consideration;  (c)  advice  is 
needed  from  other  specialists  before  a  response  can  be 
made;  (d)  request  for  a  prediction  must  be  "cannot  pre- 
dict" because  the  combinations  of  factors  extend  beyond 
our  current  level  of  understanding.  This  modeling  process 
helps  identify  topics  that  may  be  worthy  of  new  research. 

5.  Information  entered  early  in  the  process  often  ap- 
pears faulty  to  experts  when  it  is  fed  back  in  a  different 
form.  Rapid  prototyping  and  feedback  causes  respondents 
to  modify  factors,  interrelationships,  and  even  the  state- 
ment of  objectives. 

6.  This  modeling  strategy  facilitates  interdisciplinary 
research.  Individual  scientists  suggest  that  some  of  the 
uncertainty  in  their  own  models  might  be  reduced  by 
asking  advice  of  specialists  in  other  fields.  For  example, 
information  from  a  silviculturist  or  pathologist  may  help 
refine  a  critical  factor  that  an  entomologist  believes  to  be 
operating. 

7.  This  modeling  strategy  provides  a  framework  for 
integrated  data  collection.  Both  scientists  and  manage- 
ment specialists  have  observed  that  data  collection  for 
research  and  management  decision  making  can  be  more 
closely  coordinated  where  the  model(s)  in  need  of  veri- 
fication is  clearly  diagrammed. 

8.  Indirect  feedback  from  respondents  is  of  importance 
to  the  research  coordinator  and  system  designer.  Ex- 
amples include:  (a)  "this  is  too  big  a  problem  to  handle 
easily — it  should  be  stratified  into  smaller  pieces";  and 
(b)  "if  I  am  asked  to  contribute  so  much  time,  I  should 
have  joint  authorship  on  the  resultant  paper."  Comments 
such  as  these  lead  to  other,  subset,  propositions: 

A.  Knowledge  acquisition  is  complex  and  time  con- 
suming. It  places  a  time  demand  on  the  expert  that  is 
considerable.  Automated  knowledge  acquisition  proce- 
dures will  reduce  the  impacts  on  participants. 

B.  Knowledge  acquired  through  years  of  work  and 
observation  is  valuable.  The  willingness  of  experts  to 
share  that  knowledge  must  be  publicly  acknowledged 
and  rewarded. 


7 


CONCLUSIONS 

Our  work  is  directed  at  (1)  developing  a  model  or 
models  that  reflect  conditions  contributing  to  changes 
in  endemic  populations  of  mountain  pine  beetle  and 
(2)  establishing  a  foundation  for  assessing  the  role  that 
ES  programming  may  play  in  furthering  the  generation, 
clarification,  refinement,  and  verification  of  scientific 
knowledge. 

We  have  briefly  discussed  two  sets  of  propositions  that 
have  surfaced  from  our  experience  with  knowledge  acqui- 
sition and  model  building  in  a  domain  of  uncertain  and 
incomplete  scientific  knowledge  and  speculation. 

We  are  currently  at  the  stage  where  the  models  from 
different  experts  are  being  integrated.  This  task  is  being 
done  independently  by  several  investigators,  including 
several  that  were  not  involved  in  the  first  phase  of  knowl- 
edge acquisition.  The  resulting  integrated  model(s)  will 
be  distributed  to  other  MPB  specialists  throughout  the 
West  for  review  and  comment. 

It  is  now  appropriate  to  explore  opportunities  for  co- 
ordinating and  modifying  field  inventory  work  already 
planned  by  research  and  management  in  order  to  collect 
additional  data  required  for  testing  the  model(s).  The 
results  will  be  the  subject  of  future  technical  reports. 

Research  will  also  continue  on  knowledge  acquisition. 
Procedures  developed  in  this  study  are  being  refined,  the 
programs  are  being  revised,  and  we  are  applying  them 
to  other  natural  resource  problems.  Of  particular  interest 
is  the  development  of  ES  packages  that  can  be  "localized" 
by  management  specialists  to  reflect  variation  in  environ- 
mental, managerial,  social,  or  economic  condition  peculiar 
to  a  specific  area. 
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